Supplementary Materials for “Naive Parallelization of Coordinate Descent Methods and an Application on Multi-core L1-regularized Classification”

The subproblem is in a form of L1-regularized least squares. To perform a coordinate update in the CD subroutine, the training data is accessed/used in a feature-wise manner, which is the same as how data is used in CDN for problem (3.2). Thus, the applicability of the naive parallelization to the CD subroutine should hold here and the speedup is predicted to be at a comparable level. In addition, there are two other places to which we can apply the direct parallelization in newGLMNET. First, in preparation for solving (1.1) with CD, newGLMNET computes ∇L(w) and the diagonal entries of H beforehand. These operations can be parallelized directly. Second, newGLMNET requires a backtrack line search along a Newton step d where the function value decrease f(w + βd)− f(w) is computed for a given t. It is straightforward to parallelize each function evaluation through the naive approach. For more details of the newGLMNET algorithm, please see [4].